In complex software products which have many interacting capabilities such as message parsing, message tree navigation, business logic etc., it can be extremely difficult to isolate performance problems. Running one or two tests and profiling them is unlikely to show the cause of the performance problem unless it is obvious. There is also the problem that regression test suites can grow exponentially in size when more product features are included. Not all these tests can be run all the time. Together, these lead to a lengthy manual process of performance investigation when performance degradation is reported by automatic tests.
U.S. Pat. No. 6,668,340 discloses a method, system and program for determining a test case selection for a software application. Multiple risk values are assigned to each of the test eases of a software application, wherein the multiple risk values represent multiple types of risks associated with the software application not performing each function. An acceptable level of risk is designated. Only a subset of the test cases of the software are selected to test, wherein the subset comprises the test cases assigned risk values that exceed the acceptable level of risk. Such a testing system inherently does not test the entire functionality of a software product which may be a practical solution for saving time, but cannot be guaranteed to test the entire software product for degradation.